Senseval-3 task: Automatic labeling of semantic roles

The SENSEVAL-3 task to perform automatic labeling of semantic roles was designed to encourage research into and use of the FrameNet dataset. The task was based on the considerable expansion of the FrameNet data since the baseline study of automatic labeling of semantic roles by Gildea and Jurafsky. The FrameNet data provide an extensive body of “gold standard” data that can be used in lexical semantics research, as the basis for its further exploitation in NLP applications. Eight teams participated in the task, with a total of 20 runs. Discussions among participants during development of the task and the scoring of their runs contributed to a successful task. Participants used a wide variety of techniques, investigating many aspects of the FrameNet data. They achieved results showing considerable improvements from Gildea and Jurafsky’s baseline study. Importantly, their efforts have contributed considerably to making the complex FrameNet dataset more accessible. They have amply demonstrated that FrameNet is a substantial lexical resource that will permit extensive further research and exploitation in NLP applications in the future.